Dynamic content generation based on response data

ABSTRACT

Methods and systems are described for collecting response data, such as electroencephalography data, functional magnetic resonance imaging data, galvanic skin response data, heart rate data, body temperature data, eye tracking data, face tracking data, head tracking data, etc., as users receive a presentation of digital content and then utilizing that response data to dynamically produce or revise other digital content, such as advertisements, to elicit a specific user response and an expected engagement with the digital content, such as the advertisement.

PRIORITY CLAIM

This application claims priority to U.S. Provisional Patent ApplicationNo. 62/619,699, filed Jan. 19, 2018, and titled “Dynamic ContentGeneration Based on Response Data,” the contents of which areincorporated by reference herein in their entirety.

BACKGROUND

With the continued increase in mobile device usage and the availabilityto digital content, advertising is shifting from generic printadvertising to user specific and targeted digital advertising. However,this shift has resulted in advertisers having more difficulty developingtargeted advertisements for the wide variety of consumers and theirpreferences. Likewise, consumers have become more inundated withadvertisements making it even more difficult for advertisements to standout and be engaging to the consumers.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a system for communicating with userdevices, publishers, and advertisers, in accordance with describedimplementations.

FIG. 2 is a diagram illustrating a plurality of controlled groupsutilized to collect response data in a controlled environment, inaccordance with described implementations.

FIG. 3 is a more detailed view of a controlled group participant viewingdigital content and response data being collected, in accordance withdescribed implementations.

FIG. 4 illustrates user collected metrics produced in response to a userof a controlled group viewing an advertisement, in accordance withdescribed implementations.

FIGS. 5A and 5B illustrate user collected metrics produced in responseto a user of a controlled group viewing an advertisement, in accordancewith described implementations.

FIGS. 5C and 5D are graphical illustration of an advertisement, inaccordance with described implementations.

FIG. 6 is a block diagram of a machine learning system that receivesresponse data, according to described implementations.

FIG. 7 is a block diagram of the machine learning system of FIG. 6,producing output models based on the training inputs, in accordance withdescribed implementations.

FIG. 8 is a block diagram of an example trained machine learning systemthat produces recommended digital content configurations based on userinputs and the trained models, in accordance with describedimplementations.

FIG. 9 is a block diagram illustrating the exchange of informationbetween a machine learning system, an ad server system, group metrics, auser device, publishers, an advertiser, and an attributor, in accordancewith described implementations.

FIG. 10 is an example response data collection process, in accordancewith described implementations.

FIG. 11 is an example digital content presentation process, inaccordance with described implementations.

FIG. 12 is an example continuous update of dynamic digital contentprocess, in accordance with described implementations.

FIG. 13 is a pictorial diagram of an illustrative implementation of aserver system that may be used for various implementations.

DETAILED DESCRIPTION

Methods and systems are described for collecting response data, such aselectroencephalography (“EEG”) data, functional magnetic resonanceimaging (“fMRI”) data, galvanic skin response (“GSR”) data, heart ratedata, body temperature data, eye tracking data, face tracking data, headtracking data, macro-expression tracking data, micro-expression trackingdata, etc., as users receive a presentation of digital content (e.g.,images, video, audio, tactile) and then utilizing that response data todynamically produce or revise other digital content to elicit a specificuser response and an expected engagement with the digital content.

Digital content, as used herein, refers to any type of digital contentthat may be presented to a user, including but not limited to audio,video, images, tactile or haptic, etc., or any combination thereof. Insome implementations, digital content may be a digital advertisement, adigital document (e.g., whitepaper), a digital video clip, a segment ofan application, etc. Likewise, in some implementations, one or manydifferent forms of digital content may be presented to users as sensordata is collected in response to that presentation. The collectedresponse data may then be used to produce or revise other forms or typesof digital content. For example, in some implementations, users in acontrolled environment (discussed below) may be presented with a firstform of digital content, such as a series of images or colors, and theresponse data collected from sensors as the first form of digitalcontent is presented may be utilized to generate or revise a second formof digital content, such as digital advertisements.

In some implementations, one or more controlled environments may beutilized to collect response data from a group of users viewing digitalcontent in the controlled environment, referred to herein generally as acontrolled group of users or a controlled group. In such an environment,various aspects, such as the environmental conditions (lighting,temperature, noise, etc.), may be controlled as users receivepresentations of digital content. Likewise, one or more sensors, such asEEG sensors, fMRI sensors, GSR sensors, skin temperature sensors,cameras, accelerometers, gyroscopes, etc., may be used to collectresponse data for each user as the user receives the presentation.

The collected response data are representative of a user's consciousand/or non-conscious (or sub-conscious) responses to aspects orvariables included in the presented digital content. Responses mayinclude, for example, user primal responses (also referred to herein assystem 1 responses), which are responses to different colors, light, andsound, and secondary response (also referred to herein as system 2responses) which are responses produced with respect to reading,writing, and logic.

For example, EEG data from an EEG sensor may be utilized as a measure ofcognitive-affective processing in absence of behavioral responses. Asone example, peaks in brain activity indicated by the EEG sensor may beindicative of interest in a corresponding portion of digital contentthat is presented. Eye tracking data produced from video data generatedby a camera may be utilized as a measure of visual attention patterns onthe overall digital content and specific variables (e.g., branding)within the digital content. GSR data and/or body temperature data fromGSR sensors and/or temperature sensors may be utilized as a measure ofemotional arousal to different aspects or variables of the digitalcontent. Facial expression data, such as micro-expressions and/ormacro-expressions, determined from the processing of video datagenerated by a camera, may be utilized as a measure of emotionalresponses (e.g., frustration, joy, fear, anger, trust) to differentaspects or variables of the digital content. In still other examples,questionnaire data, derived from a questionnaire completed by the userafter receiving a presentation of digital content may provide indicatorsregarding content retention, user perceived likeability, etc. As will beappreciated, any number and/or type of data may be collected in acontrolled environment by one or more sensors and utilized with thedisclosed implementations.

Collected response data may be aggregated by a machine learning systemand correlated with different variables of the presented digitalcontent, other information known about the user, and behaviors orengagements with the digital content exhibited by the users and/or otherusers in response to similar variables and/or other digital content. Forexample, the collected response data, actual user engagement in responseto the presented digital content, and user profile information, such asage, demographics, location, etc., may be provided as inputs to themachine learning system. Utilizing the inputs, the machine learningsystem may be trained to produce models indicating different variables(e.g., text size, color, sounds, objects, etc.) that will produce adesired response by users of a particular user type. Likewise, thedesired response may be further correlated with an expected orlikelihood of engagement by the user. A user type, as discussed furtherbelow, may be any type or group of users having one or more commoncharacteristics. For example, a user type may be defined as users thatlive in a particular geographic area (e.g., the south), users within aparticular age range, users that respond to different variables in asimilar manner, etc.

The produced models may be used to refine or revise other existingdigital content to improve user responses to digital content and/or togenerate new digital content that will produce a desired user responseand corresponding likelihood of engagement. Because system 1 and system2 responses are at a user's core level and mostly uncontrolled orsubconscious responses to a stimulus, utilizing models produced based onactual sensor data indicative of those system 1 and/or system 2responses from users of a particular user type allows digital content tobe created that is highly tuned and able to produce the desired system 1or system 2 response from users of a particular user type. Likewise,because users often behave in a manner consistent with their system 1and system 2 responses and actual engagement has been modeled inresponse to presentations of digital content with the differentvariables (e.g., different advertising variables), likelihood ofengagement by a user can be predicted to occur from users of aparticular user type based on the digital content created according to agenerated model.

The models produced according to the described implementations provide atechnical advantage over existing systems by enabling the production ofdigital content according to those models such that the digital contentwill produce a predicted and repeatable user response from users of aparticular user type, thereby increasing the likelihood of userengagement. For example, models may be used to produce digital contentthat will elicit particular system 1 or system 2 responses from a userof a particular user type when viewing the digital content. Likewise,user engagement may also be highly correlated with the responses to thepresented digital content, thereby increasing the probability that theuser will behave (e.g., engage) in a predicted manner.

As discussed further below, any variety of inputs may be utilized withthe machine learning system to produce models. For example, any one ormore of the following inputs may be considered by a machine learningsystem in the production of a model for a particular user type: usertype information, a user profile of a particular user for which digitalcontent is to be generated and presented, a placement of generateddigital content within an application, a user device profile,environmental factors (e.g., time of day, day of week, location of theuser device), etc.

As discussed further below, in addition to collecting sensor data in acontrolled environment, in some implementations, sensor data may becollected from users in uncontrolled environments as the users arepresented with various digital content. For example, digital content,such as an advertisement, may be generated based on a model producedaccording to the described implementations and presented to a user asthe user commutes to work on the bus (an uncontrolled environment). Asthe user views the digital content, response data may be collected fromvarious sensors associated with the user, such as a camera on the user'sphone, heart rate or skin temperature measured from a wearable deviceworn by the user, etc. The collected response data, environmentalconditions present at the time the user views the digital content, andthe user's actual engagement in response to the digital content may becollected and utilized as feedback to the machine learning system as ininput for that user and/or for users of the user type corresponding tothat user. Such feedback may be utilized to adjust or refine models fordigital content generated for that user and/or for other users of thatuser type.

For ease of discussion, the following examples will be describedprimarily with respect to advertisements as digital content. However, itwill be appreciated that other forms of digital content may be utilizedwith the described implementations as part of the presentation to theuser for collection of sensor data and/or as digital content that isrevised or created based on produced models. For example, the describedimplementations may be utilized to collected response data as a firsttype of digital content is presented to users in a controlled and/oruncontrolled environment, produce models based on that response data,and then utilize those models to generate or revise the same ordifferent forms of digital content.

As used herein, “advertisers” include, but are not limited to,organizations that pay for advertising services including ads on apublisher network of applications and games. “Publishers” providecontent for users. Publishers include, but are not limited to,developers of software applications, mobile applications, news content,gaming applications, sports news, etc. In some instances, publishersgenerate revenue through selling ad space in applications so thatadvertisers can present advertisements in those applications to users asthe users interact with the application.

Advertisement performance can be defined in terms of one or more ofclick-through rates (CTR), conversion rates, advertisement completionrates, future transactions with the advertiser, post install actions orrevenue, etc. In some implementations, advertisement performance may bedefined in terms of response data collected from users as the user viewsthe advertisement.

The process in which a user selects an advertisement is referred to as aclick-through, which is intended to encompass any user selection of theadvertisement. The ratio of a number of click-throughs to a number oftimes an advertisement is displayed is referred to as the CTR of the ad.A conversion of an advertisement occurs when a user performs atransaction related to a previously viewed advertisement. For example, aconversion may occur when a user views an advertisement and installs,within a defined period of time, an application being promoted in theadvertisement. As another example, a conversion may occur when a user isshown an advertisement and the user purchases an advertised item on theadvertiser's web site within a defined time period. Except whereotherwise noted, click-through, conversion, or other positive engagementby a user with an advertisement is generally referred to herein as an“engagement.”

The ratio of the number of conversions or engagements to the number oftimes an advertisement is displayed is referred to as the conversionrate. A completion rate is a ratio of a number of video ads that aredisplayed to completion to a number of video ads initiated on a device.In some examples, advertisers may pay for their advertisements throughan advertising system in which the advertisers bid on ad placement on acost-per-click (CPC), cost-per-mille clicks (CPM),cost-per-completed-view (CPCV), cost-per-action (CPA), and/orcost-per-install (CPI) basis. A mille represents a thousand impressions.

FIG. 1 is a block diagram of an ad server system 100 for communicatingwith user devices 102, 104, 106, publishers 182, advertisers 184, andone or more controlled environments 185, in accordance with describedimplementations.

As illustrated in FIG. 1, the ad server system 100 includes anadvertising engine 130, processing logic 132, device profiles 134,storage medium 136, an ad store 150, an ad builder format tool 192, andan auction system 190. The auction system 190 may be integrated with thead server system 100 or separate from the ad server system 100. The adserver system 100 provides advertising services for advertisers 184 touser devices 102, 104, and 106 (e.g., source device, client device,mobile phone, tablet device, laptop, computer, wearable, connected orhybrid television (TV), IPTV, Internet TV, Web TV, smart TV, satellitedevice, satellite TV, automobile, airplane, etc.) in an uncontrolledenvironment 187. A user device profile for a device is based on one ormore parameters including location (e.g., GPS coordinates, IP address,cellular triangulation, Wi-Fi information, etc.) of the device, a userprofile for a user of the device, and/or categories or types ofapplications installed on the device. Each user device may includerespective advertising services software 103, 105, 107 (e.g., a softwaredevelopment kit (SDK)) that includes a set of software development toolsfor advertising services including in-application advertising services.The user devices may further include response software 108, 109, 110that collects sensor data from the user device and/or other user devicesassociated with the user as advertisements are presented to the user. Aswill be appreciated, users of the devices 102, 104, 106 may be in anuncontrolled environment, which, as used here, is any environment otherthan a controlled environment 185, discussed herein, in which a user mayinteract with and/or use the device 102, 104, 106. For example, anuncontrolled environment may include a user's home, a bus, a park, ashopping mall, or any other location of a user.

In comparison, as discussed further below, a controlled environment 185is a controlled setting in which advertisements are presented to usersunder known environmental conditions. For example, a controlledenvironment may be a room in which the noise, temperature, lighting,etc., is controlled along with the time of day, day of week, week ofmonth, and/or month of year in which advertisements are presented to auser and sensor data collected from sensors present in the controlledenvironment.

The publishers 182 publish content along with selling advertisementspace to advertisers. Attributers 186 may install software (e.g.,software development kits of publishers) on client devices and trackuser interactions or engagement with publisher applications and/oradvertisements. The attributers 186 may then share this user data withthe ad server system 100 and the appropriate publishers 182 andadvertisers 184. The ad server system 100, devices 102, 104, 106,advertisers 184, publishers 182, attributers 186, controlledenvironments 185, and an ad exchange 195 with third party exchangeparticipants communicate via a network 180 (e.g., Internet, wide areanetwork, WiMAX, satellite, etc.). The third party exchange 195participants can bid in real time or approximately in real time (e.g., 1hour prior to an ad being played on a device, 15 minutes prior to an adbeing played on a device, 1 minute prior to an ad being played on adevice, 15 seconds prior to an ad being played on a device, less than 5seconds prior to an ad being played on a device, less than 1 secondprior to an ad being played on a device) to provide advertising services(e.g., an in-application ad that includes a preview (e.g., videotrailer) of an application, in-application advertising campaigns forbrand and performance advertisers) for the devices.

In one example, an ad format builder tool 192 dynamically generates andprovides advertisements for insertion at an ad placement position withinan application for presentation to a user via a user device. The adformat builder tool allows a publisher or developer to create a newcustom advertisement campaign, associate content items of theadvertisement campaign with various tokens, select templates usable fordynamic creation of advertisements, etc. The ad format builder tool maythen utilize the information from the advertisement campaign inconjunction with a model generated from the described implementations todynamically produce an advertisement for a specific user that willelicit a desired response and increase a likelihood of an engagement bythe user with the advertisement.

The ad format builder tool 192 provides a technological improvement toadvertisers allowing them to provide content items for advertisementcampaigns so that advertisements can be dynamically generated accordingto a model specific to that user and the existing conditions such thatthe advertisement will produce specific system 1 and system 2 responsesfrom that user, thereby increasing the likelihood of engagement by theuser.

In some implementations, the system 100 includes a storage medium 136 tostore one or more software programs, content items, etc. Processinglogic 132 is configured to execute instructions of at least one softwareprogram to receive an advertising request from a user device 102, 104,106. An ad request may be sent by a device upon the device having an adplay event for an initiated software application and/or upon initiationof an application in which ads may be presented. The processing logic isfurther configured to send a configuration file and/or an advertisementto the device in response to the advertising request. The configurationfile may include different options for obtaining at least oneadvertisement (“ad”) to play on the user device during an ad play event.Alternatively, or in addition thereto, the configuration file mayidentify an advertisement that is stored in a memory (e.g., cache) ofthe user device, identify an advertisement to be obtained from the adstore 150 of the ad server system 100, identify or provide a dynamicallygenerated ad that is to be presented, and/or indicate that anadvertisement is to be obtained from the exchange 195.

FIG. 2 is a diagram illustrating a plurality of controlled groups 202utilized to collect response data in a controlled environment, inaccordance with described implementations. Controlled groups 202 may beestablished anywhere in the world 200 and/or in anywhere in the universethat may be populated by humans. The illustrated example, shows theUnited States of America 201 and five controlled groups 202-1, 202-2,202-3, 202-4, 202-5. A controlled group may be established at anylocation and any number of controlled groups may exist. Likewise,controlled groups may be periodically established at the same ordifferent locations to collect response data in response toadvertisements and that response data, as used herein, may be utilizedto update and revise the machine learning system, the produced models,and the ultimate advertisements.

A controlled group may include any number of users and sensor data maybe collected for those users for any length of time. In someimplementations, users of a controlled group may all be selected asusers of a particular user type. For example, the controlled group 202-5may be selected to be men living in Texas between the age of 30-45 thatare married and earn income between $40,000 USD and $83,000 USD. As willbe appreciated, any number and/or type of characteristics may be used todetermine a user type for a group of users.

FIG. 3 is a more detailed view of a controlled group participant 303 oruser of a controlled group 302 within a controlled environment viewingan advertisement 315 and response data being collected by varioussensors 304 within a controlled environment, in accordance withdescribed implementations. In this example, the user 303 is viewing anadvertisement 315 presented on a display of a user device 306 that iswithin the controlled environment. As the user views the advertisement,sensor data is collected from various sensors. Sensors may include, butare not limited to, an EEG sensor 304-1, device camera 304-2, microphone304-3, GSR sensor 304-4, vision, gaze, face and/or head tracking camera304-5, fMRI sensor, etc. In other implementations, other sensors may beused in conjunction with or as an alternative to the exemplified sensors304. Likewise, in some implementations, data may be collected from aquestionnaire 304-6 or other input/form that is completed by the user303 after the user has viewed the advertisement.

As the user 303 views the advertisement and sensor data are collectedfrom the various sensors 304, the sensor data is correlated in time withthe presentation of the advertisement. Based on the time correlationbetween the different sensor data and the advertisement, user responsesmay be determined and the advertisement variable present at the time ofthe determined responses identified.

For example, FIG. 4 illustrates user collected sensor data, alsoreferred to herein as response data, produced in response to the user303 viewing the advertisement 315, in accordance with describedimplementations. In this example, the user interacts or plays anapplication 410 and is periodically presented with advertisements 415-1,415-2-415-N at various ad placement positions within the advertisement.As will be appreciated, any number and/or type of advertisements may bepresented to the user. As the user view the advertisements and/orinteracts with the application, sensor data may be collected from thevarious sensors and time correlated with the presented advertisement orapplication. For example, sensor data visualization 402-1 provides avisual illustration of the sensor data collected as the advertisement415-1 is presented to the user. For example, graph 406-1 is avisualization of the sensor data collected over time 401 from the EEGsensor 304-1, graph 406-2 is a visualization of the sensor datacollected over time 401 from the device camera 304-2, graph 406-3 is avisualization of the sensor data collected over time 401 from themicrophone 304-3, graph 406-4 is a visualization of the sensor datacollected over time 401 from the GSR sensor 304-4, and graph 406-5 is avisualization of the sensor data collected over time 401 from the camera304-5. In some implementations, sensor data may be collected before,during, and/or after the advertisement. For example, sensor data may becollected for a defined period of time before the advertisement toestablish a baseline for the user and/or to determine environmentalconditions before the advertisement is presented. Likewise, sensor datamay be collected after completion of the advertisement to monitor longterm influence of the advertisement on the user.

Based on the visualization 402-1 of the sensor data, particular pointsin time, such as point 403 may be identified in which the sensor datacollectively indicates a particular response from the user. For example,at point 403 the sensor data 406-1, 406-2, 406-3, 406-4 indicate achange or peak in activity by the user at that point in time of theadvertisement and it may be determined from that combination of sensordata that the user has presented a response of interest to theadvertisement. Likewise, the presented segment 500 of the advertisementmay be updated to correspond to a selected point in time, such as point403. Referring briefly to FIG. 5A, illustrated is a detail view of thesegment 500 of the advertisement corresponding to point 403. In thissegment illustrated is an airplane 506 in the center of theadvertisement and a brand name 508 presented in the lower corner of theadvertisement. Overlaid on the segment 500 are visual identifiers or hotspots 502, 504, determined based on a processing of the sensor data(video) from the device camera 304-2 or the camera 304-5. The hot spots502, 504 are indicative of a user gaze direction, determined from eyetracking, during this segment 500 of the advertisement. Based on thisvisualization, it may be further determined that, in this example, thepresentation of both a main item, such as the airplane 506 and asecondary item, such as the brand name 508 may cause distraction tousers viewing the advisement, as indicated by the variation in the eyetracking between the two presented items. Such information may be usedalone, for example to refine an existing advertisement, and/or, asdiscussed below aggregated with other sensor data from other usersviewing the advertisement as part of a machine learning system input toproduce a model.

Returning to FIG. 4, visualization 402-2 presents another view of sensordata over time and corresponding to different segments of theadvertisement 415-1. In this example, the different sensor data isvisualized as line graphs 416-1, 416-2, and 416-3 and different segments425-1, 425-2-425-N may be presented along with the visualization of thesensor data. In this example, rather than presenting the sensor dataindividually, one or more items of sensor data may be combined, and linegraphs produced indicating a response that can be determined from acombination of that sensor data. For example, line graph 416-1 may be acombination of the sensor data indicative of system 1 responses tocolors, sights, and sounds and line graph 416-2 may be a combination ofsensor data indicative of system 2 responses to text, language, andlogic.

FIG. 5B illustrates yet another example visualization 504 of userresponses, such as neutral, happy, anger, feature, other, as determinedfrom sensor data collected in a controlled environment as the userreceives a presentation of an advertisement.

Collection of sensor data within a controlled environment reduces orholds approximately constant the influence of other variables (e.g.,noise, temperature, lighting) that are external to the application sothat a direct correlation between the user responses, as measured by thesensor data, and variables (color, text size, sound, etc.) of thepresented advertisements can be measured.

FIG. 5C is a graphical illustration of an advertisement 520 that resultsin high cognitive load of a user, as indicated by sensor data from anEEG sensor. In comparison, FIG. 5D is a graphical illustration of thesame advertisement 520 modified based on output from the machinelearning system to reduce the amount of text presented to the user,resulting in reduced cognitive load and higher user engagement, inaccordance with described implementations.

In the illustrated example, the advertisement 520 of FIG. 5C includesthe text “Have Your Character Blast The Cubes” 523-1, which isdetermined by the described implementations to reduce user engagementand require high cognitive load by the user to process the information.In comparison, the same advertisement 520 of FIG. 5D with the modifiedtext “Blast The Characters!” 532-2, produced based on a configurationfile output from the trained machine learning system to simplify thetext and increase the size of the font, results in higher userengagement and reduced cognitive load on the user viewing theadvertisement.

In the illustrated examples, the client device presents on a display 527of the device the advertisement 520. The advertisement, which in thisexample includes the cubes to be blasted, is presented on the displayand may be interacted with by the user. Overlaid on the advertisement520 is a call to action 523 instructing the user how to interact withthe advertisement. The call to action 523-1 presented in FIG. 5C is longand wordy, resulting in higher cognitive load requirements of theviewing user, compared to the simplified call to action 523-2 presentedin FIG. 5D, which results in higher user engagement.

As discussed herein, the described implementations are operable todynamically generate a variety of advertisements for an ad campaign forpresentation to a user that is accessing an application on a clientdevice. While the above example illustrate advertisements that includemodified text and font size, in other implementations other variablesmay be changed. For example, the colors of the advertisement, charactersin the advertisement, duration of the advertisement, sounds included inthe advertisement, font type, font treatment, text content, an objectincluded in the advertisement, a video type, a character type, ananimation, an interactive element (e.g., character, object), aninteraction complexity, an image, a haptic output, a brightness, acontrast, etc., may be modified or adjusted based on a configurationfile produced from the trained machine learning system.

FIG. 6 illustrates a plurality of users 601(1), 601(2), 601(3), -601(N)that interact with applications and/or view digital content, such asadvertisements, presented on a respective user device and havecorresponding user profiles 610-1, 610-2, 610-3, -610-N. As the users601 interact with digital content presented to the users, actual sensordata is collected and included as part of response profile 650-1, 650-2,650-3, and 650-N. In addition, other data, generally referred to hereinas conditions 609, may also be included in the response profile asactual conditions at the time the digital content is presented to theuser. For example, the device profile (e.g., device type, orientation,connectivity, etc.), and/or environmental conditions (e.g., device/userlocation, time of day, day of week, temperature, lighting, ambientnoise) may be determined and included in each response profile 650.

For example, when user 1 601-1 is accessing an application and ispresented with digital content, the resulting response profile mayinclude sensor A data 603-1, sensor B data 604-1, sensor C data 605-1,sensor D data 606-1, sensor E data 607-1, and indication of the digitalcontent 608-1 that was presented, the conditions 609-1 under which theuser 601-1 viewed the digital content, and the user profile 610-1corresponding to the user 601-1. In addition, an actual user responseand/or engagement may likewise be determined and stored as part of theresponse profile 650-1 for the user 601-1.

In a similar manner, when user 2 601-2 is accessing an application andis presented with digital content, the resulting response profile mayinclude sensor A data 603-2, sensor B data 604-2, sensor C data 605-2,sensor D data 606-2, sensor E data 607-2, and indication of theadvertisement 608-2 that was presented, the conditions 609-2 under whichthe user 601-2 viewed the advertisement, and the user profile 610-2corresponding to the user 601-2. In addition, an actual user response orengagement may likewise be determined and stored as part of the responseprofile 650-2 for the user 601-2. In a similar manner, when user 3 601-3is accessing an application and is presented with the digital content,the resulting response profile may include sensor A data 603-3, sensor Bdata 604-3, sensor C data 605-3, sensor D data 606-3, sensor E data607-3, and indication of the advertisement 608-3 that was presented, theconditions 609-3 under which the user 601-3 viewed the advertisement,and the user profile 610-3 corresponding to the user 601-3. In addition,an actual user response or engagement may likewise be determined andstored as part of the response profile 650-3 for the user 601-3. In asimilar manner, when user N 601-N is accessing an application and ispresented with digital content, the resulting response profile mayinclude sensor A data 603-N, sensor B data 604-N, sensor C data 605-N,sensor D data 606-N, sensor E data 607-N, and indication of theadvertisement 608-N that was presented, the conditions 609-N under whichthe user 601-N viewed the advertisement, and the user profile 610-Ncorresponding to the user 601-N. In addition, an actual user response orengagement may likewise be determined and stored as part of the responseprofile 650-N for the user 601-N.

As will be appreciated, each of the users 601-1, 601-2, 601-3 through601-N may be in a controlled environment, in which case the conditions609 are known and controlled, or an uncontrolled environment, in whichcase the conditions may be measured by one or more sensors or otherwisedetermined (e.g., from a third-party service).

As the response profiles are generated with actual informationcorresponding to digital content, the response profiles are provided toa machine learning system 655 as training inputs.

Because users vary in their preferences, interests, application usepatterns, behaviors, etc., and those things further vary based on otherconditions (e.g., time of day, day of week, time of year, location,weather, etc.), actual user responses (e.g., system 1 and system 2) andresultant engagements exhibited by users in response to differentdigital content with different variables (e.g., color, sound, text size,objects) will likewise vary. As a particular example, if anadvertisement, such as the advertisement discussed above with respect toFIG. 4 and FIGS. 5A-5B is presented to three different users at the sameadvertisement placement position within the same application and withinthe same controlled environment, the responses (system 1 and system 2)from each user may vary.

In accordance with the present disclosure, the response profile, such asthe collected sensor data discussed above with respect to FIG. 4 and theother aspects of a response profile discussed with respect to FIG. 6 maybe provided as inputs to a machine learning system 655, either in realtime or in near-real time, as the data is collected in response topresented digital content. In addition to the training inputs, in someimplementations, actual user engagements by other similar users inresponse to the presented digital content may be provided to the machinelearning system.

The machine learning system 655 may be fully trained using a substantialcorpus of response profiles, digital content, and/or engagement withthat digital content to develop models for different user types underdifferent conditions. For example, a user type may include users havingsimilar user profiles and that produce similar responses to segments orforms of digital content presented under the same or similar conditions.Such types of users may be correlated and associated as a user type bythe machine learning system 655 and utilized as representative of usersthat are similar. Likewise, after the machine learning system 655 hasbeen trained, and the models developed, the machine learning system maybe provided with a user profile, current conditions, applicationinformation, digital content placement position information, and adevice profile, and the machine learning system will generate a modeland corresponding engagement prediction that is to be used to produce orselect digital content, such as an advertisement, to present to theuser.

Training of the machine learning system may include thousands ormillions of response profiles and corresponding conditions, digitalcontent (e.g., advertisements), users, etc., under controlled and/oruncontrolled conditions.

Referring to FIG. 7 illustrated are example models 700 that may beoutput by the machine learning system 755 in response to receivingtraining inputs. For example, the training inputs, as discussed abovewith respect to FIG. 6, may include multiple different response profilesfor multiple different items of digital content and users undercontrolled and/or uncontrolled conditions. In some implementations, themachine learning system may develop models for different groups of users(i.e., user types) and/or for varying conditions, such as differentapplications, different digital content placements within anapplication, time, day, weather, location, etc.

The models 706 produced by the machine learning system may classify usertypes and conditions into different models, such as model 1 700-1, model2 700-2, -model N 700-N. Each model may be associated with differentuser types and/or varying conditions and producing similar likelihood ofengagement under those conditions. In some implementations, a model maybe associated with or representative of hundreds, thousands, or millionsdifferent user types and condition pairs. Likewise, in someimplementations, a response profile may correspond to one and only onemodel. In other implementations, a response profile may correspond todifferent models, for example, depending on the desired engagement,desired response, and/or the conditions.

Included in each model is an indication of the user type (e.g.,demographics, age range, device types, etc.) corresponding to the modeland representative of the response profiles utilized to develop themodel 700. For example, model 700-1 includes user types 702. Likewise,each model may indicate one or more conditions 704 to form a user typeand condition pair, and an engagement prediction 706 for the user typecondition pair when digital content, such as an advertisement,corresponding to the model is presented to a user of that user typeunder similar conditions. As discussed above, the conditions mayindicate, for example, any one or more of weather, location, time ofday, day of week, week of month, month of year, activity, application,digital content placement position, etc.

The models also include predicted engagement values 706 for eachcombination of user type 702 and condition 704. The predicted engagementindicates the probability of a user engagement (e.g., click-through,purchase, interaction, etc.) in response to a digital content presentedto a user similar to the determined user type with the other conditionsbeing present.

Those of ordinary skill in the pertinent arts will recognize that anytype or form of machine learning system (e.g., hardware and/or softwarecomponents or modules) may be utilized in accordance with the presentdisclosure. For example, one or more machine learning algorithms ortechniques, including but not limited to nearest neighbor methods oranalyses, artificial neural networks, conditional random fields,factorization methods or techniques, K-means clustering analyses ortechniques, similarity measures such as log likelihood similarities orcosine similarities, latent Dirichlet allocations or other topic models,or latent semantic analyses may be utilized alone or in combination todevelop user models, predict likelihood of user engagement and userresponses to digital content generated according to variables specifiedin a model.

In some implementations, a machine learning system may identify not onlya predicted engagement but also a confidence interval, confidence levelor other measure or metric of a probability or likelihood that thepredicted engagement will be exhibited by a user that is similar to theuser type in response to particular digital content generated accordingto the model and presented to the user under particular conditions.Where the machine learning system is trained using a sufficiently largecorpus of response profiles, user information, conditions, recommendeddigital content, actual user engagement, etc., the confidence intervalassociated with the predicted engagement may be substantially high.

Referring to FIG. 8, illustrated is the trained machine learning system855 receiving inputs 800 of a user profile for user A, device profile,conditions at the location of the device, and application informationindicating an application through which digital content is to bepresented. The conditions may be provided by the device and include, forexample, location of the device, orientation of the device, altitude,temperature, etc. Alternatively, or in addition thereto, conditions maybe obtained from a third-party service. For example, the machinelearning system 855 may receive location information for the device andthen obtain condition information (e.g., weather) for that location froma third party. In addition to receiving inputs 800, the machine learningsystem may determine or receive candidate digital item components, suchas advertisement components, that have been provided by a third party,such as an advertiser, for generating an item of digital content. Thedigital item components may include or identify different componentsthat may be included in an item of digital content (e.g., anadvertisement), such as different font sizes, different sounds and/oramplitude of sounds, different colors, different images, differentvideos, different objects, different haptics, different durations,different image types, different animations, different interactiveelements and/or the complexity of the interaction, different hapticoutputs, brightness, contrast, etc.

In addition, a model 802-1 may be determined based on a combination ofone or more of the inputs 800 and the candidate advertising components802-2. For example, a model that includes a user type and condition pairthat most similarly matches the user profile and conditions of theinputs may be determined and selected as the model 802-1.

Utilizing the model, the inputs 800, and the candidate digital itemcomponents, the machine leaning system determines a recommended digitalcontent configuration 804 for use in generating digital content forpresentation to the user. In response, a recommendation system, such asan advertisement recommendation system, may select or generate digitalcontent corresponding to the recommended digital item configuration andsend the recommended digital content to a user device in use by the userfor presentation to the user.

While the above examples discuss the use of a machine learning system855 for generating a recommended digital content configuration, such asa recommended advertisement configuration, based on candidate digitalitem components, in other implementations, the machine learning systemmay generate digital content variables, such as advertisement variables,that, if incorporated into the creation of a digital content willproduce a desired response and predicted engagement. For example, themachine learning system may receive the inputs 800 and an indication ofone or more desired responses and select a model that most closelycorresponds to the inputs 800 and desired response. Based on the model,digital content variables (e.g., color, speed of video, size and/or typeof images, animations, characters, haptics, font, sound, objects, etc.)may be indicated in the configuration file as recommended variables foruse in generating digital content that will produce the desired responsefrom a user of a particular user type and under the input conditions.

For example, the configuration file for a particular user or group ofuser may indicate a model for the user or group of users, digitalcontent variables, such as but not limited to, colors or color contrastsbetween colors, brightness, font size, font type, font treatment, amountof text to include in the advertisement, text placement, sounds,haptics, duration, inaction, interaction complexity, contrast, etc.,that may be utilized in dynamically generating an advertisement and/orselecting an existing advertisement, etc.

In still another example, the machine learning system may receive theinputs 800, determine an appropriate model based on the inputs 800, andselect an existing item of digital content, such as an advertisement,that most closely corresponds to the determined model, user type andconditions to elicit a desired response from a user of the user typewhen the selected digital content is presented to the user.

FIG. 9 is a block diagram illustrating the exchange of informationbetween an ad server system 900, a user device 930, publishers 982, anadvertiser 984, and an attributor 986, in accordance with describedimplementations. As with other examples discussed herein, while thisexample is described with respect to advertisements, it will beappreciated that the described implementations are equally applicable toother forms of digital content.

In the example illustrated in FIG. 9, a user device 930 (e.g., sourcedevice, client device, mobile phone, tablet device, laptop, computer,wearable, connected or hybrid television (TV), IPTV, Internet TV, WebTV, smart TV, etc.) initiates a software application (e.g., a mobileapplication, a mobile web browser application, a web based application,non-web browser application, etc.). For example, a user may select oneof several software applications 931 installed on the user device. Theadvertising services software 940 operating on the user device 930, isalso initiated upon the initiation of one of the software applications931. Likewise, in some implementations, one or more sensors 950 of thedevice and/or in communication with the device (e.g., sensors of awearable device worn by the user that is in communication with thedevice 930) may also be initiated and used to collected sensor datarelating to the user and/or the environment.

The advertising services software 940 may be associated with or embeddedwith the software application. The advertising services software 940 mayinclude or be associated with logic 942 (e.g., communication logic forcommunications such as an ad request), an ad cache store 944 for storingone or more of the ads provided by the ad server system 900 to the userdevice 930, ad streaming functionality 946 for receiving, optionallystoring, and playing ads streamed from the ad server system 900, anddevice functionality 948 for determining device and connectioncapabilities (e.g., type of connection (e.g., 4G LTE, 3G, Wi-Fi, WiMAX,etc.), bandwidth of connection, location of device, type of device,display characteristics (e.g., pixel density, color depth, etc.). Theinitiated software application(s) or advertising services software 940may have an ad play event for displaying or playing an ad on the displayof the device 930. At operation, the ad services 940 generates and sendsto the ad server system 900 a configuration call request 950. Theconfiguration call may include information about the user accessing thesoftware application, user device information, location, sensor datacollected from the sensors 950 and/or other information collected by thelogic 942.

The ad server system, in response to receiving a configuration call 950,determines options for ad play events corresponding to the softwareapplication being accessed on the user device 930 to be utilized topresent advertisements to the user. The options may include, but are notlimited to, options for obtaining an advertisement to present on theuser device in response to an ad play event, ad placement positions thatare to be utilized within the software application to presentadvertisements to the user, etc. In one implementation, a first optionmay include playing at least one ad that is cached in the ad cachememory 944 of the user device 930 during the ad play event. A secondoption may include generating or selecting an ad that corresponds to adetermined model, as discussed above. If the ad server system generatesand sends another ad in a timely manner (e.g., in time for a predictedad play event at a selected ad placement position, within a time periodset by the at least one configuration file) then the provided ad will bepresented during the predicted ad play event at the ad placementposition within the software application 931. Another option may includestreaming at least one ad to be played during the predicted ad playevent to the device 930. The configuration file can be altered by the adserver system 900 or the user device 930 without affected theadvertising services software 940.

As a user is accessing the software application 931, an ad placementposition at a future location within the software application that hasbeen indicated as an ad placement position to be utilized is determinedand the ad services 940 sends an ad request 954 to the ad server system900 requesting an advertisement to present at that ad placementposition. The ad server system 900, upon receiving the ad request 954,utilizes the information from the configuration file and optionally anyadditional information included in the ad request 954, information fromthe attributer 986, and/or information from the advertiser 984 todetermine a model and dynamically generate and send an advertisement tothe user device 930 for presentation at the ad placement position inresponse to the ad play event. Dynamic generation of an ad is discussedabove.

The ad request 954 may include different types of information,including, but not limited to, publisher settings (e.g., a publisher ofthe selected software application), an application identifieridentifying the software application 931, ad placement positioninformation for timing placement of an ad in-app, user profileinformation, user device characteristics (e.g., device id, OS type,network connection for user device, whether user device is mobiledevice, volume, screen size and orientation, language setting, etc.),environmental conditions (e.g., geographical data, location data, motiondata, such as from an accelerometer or gyroscope of the user device,weather), language, time, application settings, demographic data for theuser of the device, access duration data (e.g., how long a user has beenusing the selected application), and cache information. The ad serversystem 900 processes the ad request 954, and optionally otherinformation, as discussed herein, to determine an appropriate model andeither dynamically generate or select and existing advertisement that issent to the user device as the ad response 960 for presentation at thead placement position.

Attributers 986 may have software (e.g., an SDK of the publisher of theapplication) installed on the user's device and obtain third party userdata from the user device. This user data may include, but is notlimited to, a user's interaction and engagement with the softwareapplication, a length of time that the application is installed, anamount of purchases from the application, buying patterns in terms ofwhich products or services are purchased and when these products orservices are purchased, engagement with advertisements presented inapplications, access duration by a user of an application, etc. As theattributor collects data from the user device, the collected data may beprovided to a machine learning system 965, which may be included in thead server system or separate from the ad server system. In someimplementations, data may also or alternatively be provided directlyfrom the user device to the machine learning system 965.

In some implementations, advertisers 984, the ad server system 900,controlled group response data 920 and/or uncontrolled group responsedata may also provide data to the machine learning system 965. Forexample, the ad server system 900 may provide information indicatingcandidate advertisement components, and/or ad placement positions usedfor generating and presenting ads to a particular user in a particularapplication, etc. Likewise, in some implementations, publishers 982 mayalso provide information to the machine learning system, including, butnot limited to, application information (e.g., type and/or content ofthe application), candidate ad placement positions, etc. Still further,the machine learning system may obtain data from a user profile datastore 918-4 corresponding to a user associated with the device 930and/or obtain information from the response data store 918-5. The userprofile information maintained in the user profile data store mayinclude any information about the user associated with the device 930.Response data included in the response data store may include historicalresponse data corresponding to the user associated with the device 930and/or response data corresponding to other users that are determined tobe similar to the user interacting with the device 930.

The machine learning system 965, as discussed above, upon receivinginformation from the ad server system 900, device 930, publisher 982,attributors 986, advertisers 984, controlled groups 920 and/oruncontrolled groups 921, may generate models representative ofadvertisement variables that may be used to produce advertisements toelicit an intended user response and an expected engagement. The modelsmay be generated based on actual sensor data collected under controlledand/or uncontrolled conditions and indicate advertising variables that,if incorporated into an advertisement, will produce a desired responsefrom the user when viewed under the provided conditions. Machinelearning and the generation of models is discussed above with respect toFIGS. 6-8.

FIG. 10 is an example response data collection process 1000, inaccordance with described implementations. The example process 1000 maybe performed in a controlled environment or an uncontrolled environment.The example process 1000 begins by determining a user profile of a userfor which response data is to be collected, as in 1002. Any variety oftechniques may be used to determine a user profile. For example, theuser may provide identifying information, such as a user name andpassword, biometrics, visual identifier, etc. In other implementations,if the user does not have a user profile or is not known to the system,the user may provide user information necessary to produce a userprofile.

To collect response data, digital content, such as an advertisement, ispresented to the user, as in 1004. Presentation of a digital content maybe an audible presentation, video presentation, tactile presentation, orany combination thereof. For example, one component of an advertisement(digital content) may be visually presented on a touch-based display ofa portable device, another component of the advertisement may be audiblypresented using a speaker of the portable device, and a third componentof the advertisement may be haptically presented using an actuator ofthe portable device.

As the digital content is presented to the user, response data iscollected by one or more sensors (e.g., EEG, fMRI, GSR, microphone,camera, accelerometer) as representative of the user's response to thepresentation of the digital content, as in 1006. As discussed above, thecollected response data is time correlated with the presentation of thedigital content such that components of the digital content relate topoints in time, as in 1008. As illustrated above, the time correlateddigital content and response data may be presented visually to indicatepeaks and valleys in the collected data and corresponding to differentsegments of the digital content.

In addition, response data from the user is aggregated with responsedata from other users viewing the same and/or different digital content,as in 1010. The aggregated response data may be utilized to determinesimilar variables among the presentations that produce similar responsesfrom similar users and/or dis-similar users. For example, some variables(color, sound, images, objects) may produce a specific response for onlycertain user types while other variables may produce a similar responsefrom all users, regardless of type. By aggregating response data from alarge corpus of users the different responses from similar ordis-similar users may be identified and correlated with the respectivevariables.

Finally, based on the aggregated response data and the determinedcorrelations, utilizing a machine learning system, as discussed above,one or more models are produced or updated based on the information, asin 1012. Generation of models based on machine learning is discussedabove.

FIG. 11 is an example digital content presentation process 1100, inaccordance with described implementations. The example process 1100begins by determining a user type for a user to which digital content,such as an advertisement, is to be presented, as in 1102. The user typemay be determined, for example, based on a user profile associated withthe user and/or in response to a user input or other user information.

In addition to determining the user type, one or more conditions aredetermined, as in 1104. As discussed above, conditions may include, butare not limited to, an application in which the digital content is to bepresented, a device type, device orientation, weather, location, time ofday, day of week, week of month, month of year, etc.

Based on the user type and the conditions, a model generated by themachine learning system, discussed above, is selected that is mostlikely to produce a desired response and engagement, as in 1106. Forexample, the models may be maintained in a model data store and queriedbased on the user type and conditions to determine a model that bestmatches the user type and condition pair.

Based on the determined model, digital content is generated or selectedthat includes variables indicated in the model that, when presented tothe user, will produce the desired response and increase the likelihoodof engagement with the digital content, 1108. Finally, the generated orselected digital content is presented to the user, as in 1110.

FIG. 12 is an example continuous update of dynamic digital contentprocess 1200, in accordance with described implementations. The exampleprocess 1200 beings with the presentation of digital content to a userduring a session, as in 1202. A session may be any discrete amount oftime during which a user is accessing a device through which digitalcontent is to be presented. For example, the session may be a period oftime that the user is accessing an application through whichadvertisements (digital content) are to be presented. As anotherexample, the session may be 8:00-14:00, or any other period of time.

As the digital content is presented, response data is collected usingone or more sensors and user engagement with the digital content is alsodetermined, as in 1204. As discussed above, response data may includesensor data from any of a variety of sensors that are on the userdevice, in communication with the device, or for which sensor data isotherwise accessible. For example, if the presentation is in acontrolled environment, some or all of the response data may becollected via sensors that are independent of the device through whichthe digital content is being presented.

The response data and determined engagement may then be used to updatethe model for that user during that session, as in 1206. In someimplementations, the update may be specific to the user and/or thesession. For example, the response data received from the user duringthe session may be influenced by the user's current mood or attitude. Insuch an example, the response data collected during the session mayreceive a greater weight and be utilized to determine a model that isclosely correlated with the user based on the current response data. Inother implementations, the current response data may be persistent andutilized to update the models for user types that are similar to theuser.

Based on the updated model, a next item of digital content is generatedbased on the updated model such that the digital content includesvariables that are most likely to produce a desired response from theuser and resulting engagement with the digital content, as in 1208. Inaddition, a determination may be made as to whether the user is still inthe same session, as in 1210. For example, if the session corresponds tothe user's application access or period of time, it may be determinedwhether the same session exists for the user.

If it is determined that the same session exists, the example process1200 returns to block 1202 and continues by presenting the next digitalcontent to the user, as in 1210. However, if it is determined that thesession has ended, the example process 1200 completes, as in 1212. Theexample process 1200 may be performed continuously as users interactwith applications and receive digital content, such as advertisements,so that digital content can be continually revised to best match thecurrent mood of the user for which the digital content is to bepresented.

FIG. 13 is a pictorial diagram of an illustrative implementation of aserver system 1300, such as a remote computing resource, that may beused with one or more of the implementations described herein. Theserver system 1300 may include a processor 1301, such as one or moreredundant processors, a video display adapter 1302, a disk drive 1304,an input/output interface 1306, a network interface 1308, and a memory1312. The processor 1301, the video display adapter 1302, the disk drive1304, the input/output interface 1306, the network interface 1308, andthe memory 1312 may be communicatively coupled to each other by acommunication bus 1310.

The video display adapter 1302 provides display signals to a localdisplay permitting an operator of the server system 1300 to monitor andconfigure operation of the server system 1300. The input/outputinterface 1306 likewise communicates with external input/output devices,such as a mouse, keyboard, scanner, or other input and output devicesthat can be operated by an operator of the server system 1300. Thenetwork interface 1308 includes hardware, software, or any combinationthereof, to communicate with other computing devices. For example, thenetwork interface 1308 may be configured to provide communicationsbetween the server system 1300 and other computing devices, such as theuser device 1100.

The memory 1312 generally comprises random access memory (RAM),read-only memory (ROM), flash memory, and/or other volatile or permanentmemory. The memory 1312 is shown storing an operating system 1314 forcontrolling the operation of the server system 1300. A binaryinput/output system (BIOS) 1316 for controlling the low-level operationof the server system 1300 is also stored in the memory 1312.

The memory 1312 additionally stores program code and data for providingnetwork services that allow user devices and external sources toexchange information and data files with the server system 1300. Thememory also stores a data store manager application 1320 to facilitatedata exchange and mapping between the data store 1318, ad server system1305, user devices, external sources, etc. While this describesinteraction with an ad server system 1305, the describe implementations,and the described example server system 1300, may likewise be utilizedwith any form of digital content server.

As used herein, the term “data store” refers to any device orcombination of devices capable of storing, accessing and retrievingdata, which may include any combination and number of data servers,databases, data storage devices and data storage media, in any standard,distributed or clustered environment. The server system 1300 can includeany appropriate hardware and software for integrating with the datastore 1318 as needed to execute aspects of one or more applications forthe user device 1100, the external sources and/or the ad server system1305. The server system 1300 provides access control services incooperation with the data store 1318 and is able to generate digitalcontent, such as advertisements.

The data store 1318 can include several separate data tables, databasesor other data storage mechanisms and media for storing data relating toa particular aspect. For example, the data store 1318 illustratedincludes advertisements, digital content items for advertisements,models, user profiles, etc.

It should be understood that there can be many other aspects that may bestored in the data store 1318, which can be stored in any of the abovelisted mechanisms as appropriate or in additional mechanisms of any ofthe data stores. The data store 1318 may be operable, through logicassociated therewith, to receive instructions from the server system1300 and obtain, update or otherwise process data in response thereto.

The memory 1312 may also include the ad server system 1305. The adserver system 1305 may be executable by the processor 1301 to implementone or more of the functions of the server system 1300. In oneimplementation, the ad server system 1305 may represent instructionsembodied in one or more software programs stored in the memory 1312. Inanother implementation, the ad server system 1305 can representhardware, software instructions, or a combination thereof. The ad serversystem 1305 may perform some or all of the implementations discussedherein, alone or in combination with other devices.

The server system 1300, in one implementation, is a distributedenvironment utilizing several computer systems and components that areinterconnected via communication links, using one or more computernetworks or direct connections. However, it will be appreciated by thoseof ordinary skill in the art that such a system could operate equallywell in a system having fewer or a greater number of components than areillustrated in FIG. 13. Thus, the depiction in FIG. 13 should be takenas being illustrative in nature and not limiting to the scope of thedisclosure.

The concepts disclosed herein may be applied within a number ofdifferent devices and computer systems, including, for example,general-purpose computing systems, and distributed computingenvironments.

The above aspects of the present disclosure are meant to beillustrative. They were chosen to explain the principles and applicationof the disclosure and are not intended to be exhaustive or to limit thedisclosure. Many modifications and variations of the disclosed aspectsmay be apparent to those of skill in the art. Persons having ordinaryskill in the art should recognize that components and process stepsdescribed herein may be interchangeable with other components or steps,or combinations of components or steps, and still achieve the benefitsand advantages of the present disclosure. Moreover, it should beapparent to one skilled in the art that the disclosure may be practicedwithout some or all of the specific details and steps disclosed herein.

Aspects of the disclosed system may be implemented as a computer methodor as an article of manufacture such as a memory device ornon-transitory computer readable storage medium. The computer readablestorage medium may be readable by a computer and may compriseinstructions for causing a computer or other device to perform processesdescribed in the present disclosure. The computer readable storage mediamay be implemented by a volatile computer memory, non-volatile computermemory, hard drive, solid-state memory, flash drive, removable diskand/or other media. In addition, components of one or more of themodules and engines may be implemented in firmware or hardware.

Unless otherwise explicitly stated, articles such as “a” or “an” shouldgenerally be interpreted to include one or more described items.Accordingly, phrases such as “a device configured to” are intended toinclude one or more recited devices. Such one or more recited devicescan also be collectively configured to carry out the stated recitations.For example, “a processor configured to carry out recitations A, B andC” can include a first processor configured to carry out recitation Aworking in conjunction with a second processor configured to carry outrecitations B and C.

Language of degree used herein, such as the terms “about,”“approximately,” “generally,” “nearly” or “substantially” as usedherein, represent a value, amount, or characteristic close to the statedvalue, amount, or characteristic that still performs a desired functionor achieves a desired result. For example, the terms “about,”“approximately,” “generally,” “nearly” or “substantially” may refer toan amount that is within less than 10% of, within less than 5% of,within less than 1% of, within less than 0.1% of, and within less than0.01% of the stated amount.

Although the subject matter has been described in language specific tostructural features and/or methodological acts, it is to be understoodthat the subject matter defined in the appended claims is notnecessarily limited to the specific features or acts described. Rather,the specific features and acts are disclosed as exemplary forms ofimplementing the claims.

What is claimed is:
 1. A computer implemented method, comprising:collecting a plurality of response data for a plurality of users of afirst type in a controlled group as each of the plurality of usersreceive a presentation of digital content, wherein the response dataincludes at least one of an electroencephalography (“EEG”) data, afunctional magnetic resonance imaging (“fMRI”) data, a galvanic skinresponse (“GSR”) data, a heart rate data, a body temperature data, aneye tracking data, a face tracking data, or a head tracking data; timecorrelating the plurality of response data with a plurality of variablesof the digital content; determining, based at least in part on the timecorrelation, an expected user response to each of the plurality ofvariables related to the digital content; and producing an advertisementbased at least in part on the expected user response to each of theplurality of variables such that the advertisement is created to elicita desired response from a user of the first type when presented to theuser.
 2. The computer implemented method of claim 1, further comprising:producing, based at least in part on the response data and the pluralityof variables a model for users of the first type.
 3. The computerimplemented method of claim 2, further comprising: determining a usertype of a user that is to receive the advertisement; determining, basedat least in part on the user type, the model; and wherein producing theadvertisement is based at least in part on the model.
 4. The computerimplemented method of claim 2, wherein the model indicates at least onevariable that may be adjusted in the advertisement to produce a desiredresponse from users of the first type.
 5. The computer implementedmethod of claim 4, wherein the variable is at least one of a font size,a font type, a font treatment, a text content, a color, a duration, asound, an object, a video type, a character type, an animation, aninteractive element, an interaction complexity, an image, a hapticoutput, a brightness, or a contrast.
 6. A system, comprising: one ormore processors; and a memory storing program instructions that whenexecuted by the one or more processors cause the one or more processorsto at least: produce a first item of digital content according to amodel output by a machine learning system for a first user of a firsttype, wherein the model indicates at least one variable to be adjustedin the first item of digital content to cause an expected response fromusers of the first type; cause the first item of digital content to bepresented to the first user of the first type; collect response dataindicative of a response presented by the first user in response to thefirst item of digital content presented to the first user; and updatethe model based at least in part on the response data to produce anupdated model.
 7. The system of claim 6, wherein the programinstructions, when executed by the one or more processors, further causethe one or more processors to at least: produce a second item of digitalcontent according to the updated model output for a second user of asecond type, wherein the second item of digital content is a variationof the first item of digital content, varied according to the updatedmodel; and cause the second item of digital content to be presented tothe second user of the first type.
 8. The system of claim 6, wherein theprogram instructions, when executed by the one or more processors,further cause the one or more processors to at least: determine at leastone condition corresponding to the first user of the first type; providethe first type of the first user and the at least one condition to amachine learning system; and receive, from the machine learning system,the model, wherein the model is based at least in part on the first typeand the condition.
 9. The system of claim 8, wherein the programinstructions, when executed by the one or more processors, further causethe one or more processors to at least: determine a desired responsefrom the first user; and wherein the model is further produced based atleast in part on the first type, the condition, and the desiredresponse.
 10. The system of claim 6, wherein: the first user is in acontrolled environment; and the response data is collected using one ormore sensors within the controlled environment.
 11. The system of claim10, wherein the one or more sensors include one or more of: anelectroencephalography (“EEG”) sensor, a functional magnetic resonanceimaging (“fMRI”) sensor, a galvanic skin response (“GSR”) sensor, aheart rate sensor, a body temperature sensor, an eye tracking sensor, aface tracking sensor, a head tracking sensor, a camera, a microphone, apressure sensor, an accelerometer, or a gyroscope.
 12. The system ofclaim 6, wherein the response is at least one of a primal response tothe first item or a secondary response to the first item.
 13. The systemof claim 6, wherein the model is produced based on a plurality ofresponse data collected from a plurality of users of the first type. 14.A method, comprising: for each of a plurality of users: presentingdigital content to the user; collecting, with a plurality of sensors,response data indicative of a response presented by the user in responseto the digital content; time correlating the response data with thepresentation of the digital content; and producing a response profilethat includes the correlated response data with variables presented atthe correlated time in the digital content; providing each of theplurality of response profiles to a machine learning system as traininginputs to train the machine learning system to produce a trained machinelearning system; developing, with the trained machine learning system, aplurality of models, each model corresponding to a different user type;determining a user of a first user type to which an advertisement is tobe presented; providing, to the trained machine learning system, thefirst user type and candidate digital content components; determining,with the trained machine learning system, a model of the plurality ofmodels corresponds to the first user type; producing, based at least inpart on the model and the candidate digital content components, anadvertisement; and presenting the advertisement to the user.
 15. Themethod of claim 14, further comprising: determining at least onecondition corresponding to the user; and wherein the model is furtherdetermined based at least in part on the at least one condition.
 16. Themethod of claim 15, wherein the condition includes at least one of adevice type of a device, a location of the device, an altitude, atemperature, a weather, an application in which the advertisement is tobe presented, a time of day, a day of week, or a month of year.
 17. Themethod of claim 14, further comprising: collecting, from the user,response data indicative of a response presented by the user in responseto the advertisement; and providing the response data to the trainedmachine learning system.
 18. The method of claim 14, wherein theplurality of users are in a controlled environment in which at least onecondition is held constant among each of the plurality of users.
 19. Themethod of claim 14, further comprising: determining a desired responseto the advertisement; and wherein the model is further determined basedon the desired response.
 20. The method of claim 14, further comprising:determining, for the model, a likelihood of engagement with theadvertisement by the user of the first user type.